Skip to content

Conversation

@pytorchbot
Copy link
Collaborator

This PR was created by the merge bot to help merge the original PR into the main branch.
ghstack PR number: #13473 by @SS-JIA
^ Please use this as the source of truth for the PR details, comments, and reviews
ghstack PR base: https://github.com/pytorch/executorch/tree/gh/SS-JIA/291/base
ghstack PR head: https://github.com/pytorch/executorch/tree/gh/SS-JIA/291/head
Merge bot PR base: https://github.com/pytorch/executorch/tree/gh/SS-JIA/290/orig
Merge bot PR head: https://github.com/pytorch/executorch/tree/gh/SS-JIA/291/orig
@diff-train-skip-merge

ssjia added 3 commits August 18, 2025 10:28
…elines()` to `prepare()`

Title says it all; `prepare()` is a more appropriate place for this action than `prepare_pipelines()`.

## Motivation

Fix potential floating point exception (divide-by-zero) during tests.

Some tests don't call `prepare_pipelines()`, which means `execute_threshold_node_count_` is unititialized, causing a divide by zero in execute when trying to modulo with `execute_threshold_node_count_`

Differential Revision: [D80468138](https://our.internmc.facebook.com/intern/diff/D80468138/)

ghstack-source-id: 303779589
Pull Request resolved: #13478
Pull Request resolved: #13472




Allow VulkanBackend to load constant tensors from the NamedDataMap instead of the constant data section of the delegate blob.

## Motivation

This enables several key results:

* Unblocks delegate retargetability with other backends
* Allows reducing peak memory usage when loading models by freeing constant weight data as it gets moved to the GPU

## Changes

* Allow `TensorRef` to be constructed with a `FreeableBuffer` rvalue
* Add ability to load constant data from `NamedDataMap` in `VulkanBackend.cpp`
* When prepacking, free the constant data pointer once it's been copied to the staging buffer
ghstack-source-id: 303830113

Differential Revision: [D80460035](https://our.internmc.facebook.com/intern/diff/D80460035/)
Pull Request resolved: #13473



When exporting models to Vulkan backend, save constant tensors in the NamedDataMap instead of the constant data section of the delegate header.

## Motivation

Prevent screen blackout (Llama 3.2 1B) / device crash (Llama 3.2 3B) when running Llama 3.2 models on Samsung Galaxy S24. This behaviour is related to high peak memory usage when loading the model. For more information, see the top diff/PR in the stack.

## Context

This change is based on the equivalent change D70315207/#9153 in XNNPACK.
ghstack-source-id: 303830114

Differential Revision: [D80460034](https://our.internmc.facebook.com/intern/diff/D80460034/)
@pytorchbot pytorchbot requested a review from SS-JIA as a code owner August 19, 2025 02:24
@pytorch-bot
Copy link

pytorch-bot bot commented Aug 19, 2025

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/13499

Note: Links to docs will display an error until the docs builds have been completed.

⏳ 8 Pending, 1 Unrelated Failure

As of commit ab383fd with merge base 5ff0208 (image):

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

  • pull / test-binary-size-linux-gcc / linux-job (gh) (trunk failure)
    /pytorch/executorch/kernels/portable/cpu/op_stack.cpp:129:26: error: comparison of integer expressions of different signedness: ‘size_t’ {aka ‘long unsigned int’} and ‘ssize_t’ {aka ‘long int’} [-Werror=sign-compare]

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 19, 2025
Base automatically changed from gh/SS-JIA/290/orig to main August 19, 2025 03:05
@SS-JIA SS-JIA merged commit 29a8612 into main Aug 19, 2025
102 of 103 checks passed
agrima1304 pushed a commit to agrima1304/executorch that referenced this pull request Aug 26, 2025
Summary:
When exporting models to Vulkan backend, save constant tensors in the NamedDataMap instead of the constant data section of the delegate header.

## Motivation

Prevent screen blackout (Llama 3.2 1B) / device crash (Llama 3.2 3B) when running Llama 3.2 models on Samsung Galaxy S24. This behaviour is related to high peak memory usage when loading the model. For more information, see the top diff/PR in the stack.

## Context

This change is based on the equivalent change D70315207/pytorch#9153 in XNNPACK.

Test Plan:


## Memory Comparison with/without NamedDataMap

Measured VmRss using

```
uint64_t getVmRssInKB() {
  std::ifstream statusFile("/proc/self/status");
  std::string l, num;
  while (std::getline(statusFile, l)) {
    if (l.substr(0, 5) == "VmRSS") {
      size_t pos = l.find_first_of("0123456789");
      num = l.substr(pos);
      break;
    }
  }
  uint64_t vmRssInKB = std::stoi(num);
  return vmRssInKB;
}
```

P1908019767 (Meta only)

Excerpt:

```
Log 1                                             | Log 2
--------------------------------------------------|--------------------------------------------------
Memory usage before model compilation: 1115416 KB | Memory usage before model compilation: 1919228 KB
Memory usage after graph building: 1924340 KB     | Memory usage after graph building: 1924256 KB
Memory usage after graph preparation: 1798968 KB  | Memory usage after graph preparation: 1782464 KB
Memory usage prepack start: 1798968 KB            | Memory usage prepack start: 1781968 KB

Memory usage after prepack operations: 1271924 KB | Memory usage after prepack operations: 1653496 KB
```

Differential Revision: [D80460034](https://our.internmc.facebook.com/intern/diff/D80460034)

[ghstack-poisoned]
@SS-JIA SS-JIA deleted the gh/SS-JIA/291/orig branch October 15, 2025 18:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants